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Foreword 


The  detection  and  classification  of  antipersonnel  land  mine  has  become  a  topic  of  very  high 
priority  in  recent  years.  Newer  methods  employ  artifical  neural  networks  for  more  accu¬ 
rate  detection.  This  research  report  presents  a  procedure  for  landmine  detection  using  a 
complex-valued  neural  network.  This  is  because  the  acquired  data  in  the  form  of  scattering 
parameters  at  different  frequencies  are  complex-valued  and  disregard  of  the  phase  works 
against  the  proven  importance  of  phase  in  multidimensional  signals.  The  complex-valued 
backpropagation  algorithm,  clearly  and  compactly  derived,  and  its  variants  are  implemented 
on  acquired  data  to  classify  mines  of  different  types  and  shapes.  The  importance  of  noise- 
contaminated  phase  as  well  as  the  role  of  partial  phase  information  in  image  reconstruction 
is  also  investigated.  Scopes  for  further  research  in  this  promising  area  and  the  use  of  unsu¬ 
pervised  learning  in  artificial  neural  networks  in  conjunction  with  fuzzy  logic  for  improved 
system  performance  over  and  above  the  good  results  already  attained  are  substantiated. 

The  role  of  wavelet  superresolution  for  multiresolution  analysis  of  landmines  in  par¬ 
ticular  and  image  analysis  in  general  is  also  reported.  Image  sequence  superresolution  (or, 
for  brevity,  superresolution)  refers  to  methods  that  increase  spatial  resolution  by  fusing  in¬ 
formation  from  a  sequence  of  nonidentical  images,  each  uniformly  sampled,  and  acquired  in 
one  or  more  of  several  possible  ways.  This  set  of  nonidentical  images,  when  superimposed, 
form  a  bigger  image  whose  region  of  support  is  a  nonuniformly  sampled  raster.  Techniques 
to  achieve  high  resolution  (superresolution)  essentially  revolve  around  attempts  to  convert 
this  nonuniformly  sampled  image  to  an  uniformly  sampled  one.  An  image  acquisition  sys¬ 
tem  composed  of  an  array  of  sensors,  where  each  sensor  has  a  subarray  of  sensing  elements 
of  suitable  size,  has  recently  been  popular  for  increasing  the  spatial  resolution  with  high 
signal-to-noise  ratio  beyond  the  performance  bound  of  technologies  that  constrain  the  man¬ 
ufacture  of  imaging  devices.  The  technique  for  reconstructing  a  high-resolution  image  from 
data  acquired  by  a  prefabricated  array  of  multisensors  was  advanced  by  Bose  and  Boo  and 
this  work  was  further  developed  by  applying  total  least  squares  to  account  for  error  not  only 
in  observation  but  also  due  to  error  in  estimation  of  parameters  modeling  the  data. 
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1  Introduction 


A  comprehensive  REPORT  FOR  RESEARCH  conducted  during  the  period  October  1,  2000 
to  August  31,  2001  was  submitted  in  September  2001  to  the  Army  Research  Office  [1]. 
Another  PROGRESS  REPORT  plus  a  few  viewgraphs  in  MS  Power  Point  with  briefing 
comments  and  paper  references  was  submitted  to  the  Army  Research  Office  on  March  27 
[2],  This  report,  therefore,  is  the  FINAL  RESEARCH  REPORT  for  US  ARMY  GRANT 
DADD19-00- 1-0539. 

The  ideal  objectives  of  a  land  mine  are  two  fold.  First,  the  goal  is  to  cause  as 
much  damage  as  is  possible  with  the  least  amount  of  material,  and  second,  the  avoidance  of 
detection  is  crucial  so  that  the  first  goal  can  be  successfully  realized.  The  second  objective 
has  been  of  major  concern  in  the  task  of  detecting  mines.  One  of  the  more  promising  avenues 
of  research  in  this  area  involves  the  use  of  artificial  neural  networks  [3].  More  specifically 
artifical  neural  networks  have,  recently,  been  used  to  detect  landmines  from  back-scattered 
radio  frequency  data  [5] .  The  two-port  complex- valued  scattering  parameters,  which  are  the 
elements  of  a  2  x  2  scattering  matrix  that  characterizes  the  two-port  in  terms  of  reflected 
and  transmitted  powers  are  measured  at  different  selected  frequencies  [6] .  The  inputs  to  the 
network  were  the  magnitudes  of  the  four  scattering  parameters  (sn,  s 1 2 ,  S21,  and  S22)  at  the 
seven  central  frequencies  (880-1120MHz)  in  40-MHZ  intervals.  Thus,  the  phase  information 
was  not  exploited.  The  phase  contains  the  information  relating  to  the  occurrence  of  edges 
in  an  image.  This  leads  one  to  believe  that  the  use  of  phase  information  may  help  in  the 
area  of  edge  detection  of  the  landmine  so  that  a  preprocessing  step  in  [7] .  of  fourth- nearest- 
neighbor  outlier  removal  of  data  points  would  not  be  necessary  since  these  points  would 
contain  valuable  information  on  the  location  of  the  boundaries  of  a  mine.  The  importance 
of  using  the  phase  along  with  the  magnitude  is  also  natural  because  the  measured  data 
is  complex- valued.  Use  of  magnitude-only  information,  therefore,  is  incompatible  with  the 
nature  of  the  acquired  data,  especially  because  valuable  information  is  contained  in  the 
phase,  which  should  not  be  summarily  either  discarded  or  de-emphasized. 

During  the  past  decade,  the  interest  in  seeking  enhancement  of  spatial  resolution  lead- 
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ing  to  a  high-resolution  (HR)  image  from  a  sequence  of  degraded  (undersampled,  blurred, 
and  noisy)  low-resolution  (LR)  images  has  been  notable.  HR  images  are  required  in  many 
areas  including  remote  sensing,  military  surveillance  and  medical  diagnosis.  Landmine  clas¬ 
sification  can  also  be  facilitated  by  superresolution  algorithms.  It  is  usually  not  possible  at 
the  outset  to  achieve  the  desired  resolution  because  of  technology  and  cost  constraints.  For 
example,  the  technology  of  a  charge-coupled  device  (CCD)  is  limited  by  factors  like  physical 
dimension,  shot  noise  and  parasitic  effects.  In  applications  like  astronomical  imaging,  the 
reduced  size  and  weight  of  a  camera  in  a  spacecraft  or  satellite  affect  its  quality.  The  need 
to  perform  a  trade-off  between  size,  weight,  and  quality  of  the  CCD  array  necessitates  the 
design  of  superresolution  algorithms  to  obtain  the  desired  HR  image. 

This  research  focusses  on  two  superresolution  algorithms:  one  is  wavelet-based  and 
the  other  is  Delaunay  triangulation  based.  Preprocessing  is  performed  prior  to  the  im¬ 
plementation  of  the  superresolution  algorithm  in  order  to  prevent  the  singularity  problem. 
Attention  is  directed  to  the  selection  of  the  mother  wavelet  and  the  mother  scaling  function. 
Then  the  postprocessing  phase  incorporating  multiframe  noise  filtering  and  blur  removal  fol¬ 
lowing  the  implementation  of  the  superresolution  algorithm  is  developed.  The  overall  blind 
superresolution  procedure  is  illustrated  on  synthetic  sequences  and  also  on  video  (degraded 
by  unknown  blur  and  noise)  sequences  supplied  by  the  Air  Force  Research  Laboratory. 

2  List  of  Appendix 

The  following  figures  are  listed  in  the  Appendix. 

1.  Figure  1  supports  the  importance  of  phase  in  the  presence  of  noise  for  image  reconstruc¬ 
tion.  Figure  2  shows  the  overall  artificial  neural  network  model  for  object  classification 
from  partial  phase  information.  It  has  been  concluded  that  reconstruction  from  phase 
information  is  more  error-resilient  than  reconstruction  from  magnitude  information 
in  the  presence  of  additive  signal-independent  noise.  When  the  phase  information  is 
blurred  and  noise  is  added,  regularization  algorithms  improve  the  the  estimate  of  a 
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reconstructed  image  from  degraded  phase.  Details  are  available  in  the  M.  S.  thesis  of 
Hung-Sik  Kim  cited  in  the  appropriate  section  below. 

2.  Figure  3  shows  the  the  high  resolution  (HR)  reconstuction  task  decomposed  into  two 
subtasks  :  interploation  and  restoration  based  on  the  image  formation  model  adopted. 
This  image  formation  model  is  used  for  atmospheric  and  motion  blur  (due  to  relative 
motion  between  object  and  image).  The  blurring  process  in  this  model  occurs  prior 
to  geometric  transformations  that  are  used  to  characterize  video  camera  motion  like 
translation,  zooming,  panning,  tilting,  and  rotation.  The  procedure  for  estimating 
the  blurred  HR  image  is  called  interpolation.  The  interpolated  image  could  be  noise 
filtered  using  a  simple  but  effective  multiframe  noise  filtering  scheme  proposed  in  this 
research.  The  case  of  unknown  blur  leads  to  the  important  practical  case  of  robust 
blind  superresolution,  also  initiated  in  this  research.  Superresolution  methods  based 
on  first  generation  wavelets  like  B-splines  and  also  the  spatial  tesselation  scheme  of 
Delaunay  triangulation  are  developed  and  implemented. 

3.  Figure  4  show  typical  frames  in  a  video  sequence  provided  by  the  Air  Force  Research 
Laboratory  (AFRL),  Rome,  New  York.  The  blur  and  noise,  that  are  sources  of  degra¬ 
dation,  are  unknown.  Figure  5  is  the  videomosaic  created  from  research  under  the 
sponsorship  of  an  AFRL  grant  following  camera  motion  parameter  estimation  using 
the  projective  model  [8].  Two  regions  of  interest  (ROIs)  are  identified  in  Figure  5. 
Figures  6  and  7  compare  the  high  resolution  image  of  the  ROIs  with  the  low  resolution 
counterparts  based  on  blur  identification,  filtering  and  superresolution  by  the  method 
of  Delaunay  triangulation  developed  in  this  research.  Details  are  available  in  the  Ph.D. 
dissertation  of  Surapong  Lertrattanapanich  cited  in  the  appropriate  section  below  and 
in  the  peer-reviewed  journal  paper  [9]  . 

4.  The  spatial  resolution  of  an  image  is  often  determined  by  imaging  sensors.  In  a  charge- 
coupled  device  (CCD)  camera,  the  image  resolution  is  determined  by  the  size  of  its 
photo-detector.  Although  a  CCD  camera  for  HD  (High  Definition)  images  has  already 
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been  developed,  it  is  still  necessary  to  increase  the  resolution  further  for  SHD  (Super 
High  Definition)  images.  Reducing  the  size  of  pixels  (photo-detectors)  is  one  way  to 
increase  resolution.  However,  the  smaller  the  pixel  size  is,  the  smaller  is  the  amount  of 
light  available  for  each  pixel  and  the  picture  quality  is  degraded  because  the  existence 
of  shot-noise  (causing  variation  of  input)  is  unavoidable.  Therefore,  a  new  scheme  is 
needed  to  synthesize  high  resolution  images,  beyond  the  physical  device  performance 
bound,  by  incorporating  signal  processing  techniques.  An  alternative  way  is  the  use 
of  prefabricated  multiple  identical  image  sensors  shifted  from  each  other  by  subpixel 
displacements  as  shown  in  Figure  10  and  then  reconstruct  a  high  resolution  image 
from  multiple  degraded  low  resolution  images  acquired  from  the  multisensor  array. 
The  relationship  between  low  resolution  image  sensor  and  hypothetical  high  resolution 
image  sensor  is  shown  in  Figure  11  and  an  example  is  shown  in  Figure  12.  Details 
are  available  in  the  Ph.D.  dissertation  of  Jaehoon  Koo  cited  in  the  appropriate  section 
below  and  in  the  peer-reviewed  journal  papers  [10],  [10].  . 


3  Statement  of  the  Problem  Studied 

In  the  Fourier  representation  of  signals,  spectral  magnitude  and  phase  tend  to  play  different 
roles  and  in  some  situations,  many  of  the  important  features  of  a  signal  are  preserved  if  only 
the  phase  is  retained  [4],  For  examples,  both  phase-only  and  magnitude-only  acoustic  and 
optical  holograms  have  been  studied.  For  phase-only  hologram  [4]  (also  referred  to  as  kino- 
forms)  only  the  phase  of  the  scattered  wavefront  is  recorded  and  the  magnitude  is  assumed 
to  be  constant  while  in  the  magnitude-only  hologram  the  phase  is  assumed  to  be  zero  and 
only  the  magnitude  of  the  scattered  wavefront  is  recorded. 

Irregular  (nonuniform)  sampling  theory  and  its  numerical  implementation,  important 
in  superresolution  from  a  sequence  of  low  resolution  discrete  signals,  is  well  developed  in  the 
1-D  case  due  to  the  work  of  Kadec  and  others  [12].  Results  in  the  area  of  efficient  robust 
reconstruction  from  irregular  samples  in  the  multidimensional  case  are  of  more  recent  vintage 
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and  even  the  generalization  of  1-D  nonuniform  sampling  results  to  n-D  is  far  from  being 
straightforward.  The  hrst  generation  wavelet  basis  for  reconstruction  from  nonuniformly 
sampled  data  was  initially  discussed  in  [13]  and  the  importance  of  the  selection  of  mother 
wavelet  and  scaling  function  (generated  by  discrete  filter  bank)  has  been  demonstrated  in  [9] . 

In  the  high  resolution  reconstruction  with  multisensor  array  system,  speed  is  impor¬ 
tant  as  well  as  the  quality  of  the  reconstructed  image.  The  desired  algorithm  should  be 
computationally  efficient  and  process  data  as  they  are  received.  Another  goal  of  this  re¬ 
search  is  to  investigate  various  aspects  of  high  resolution  image  reconstruction  like  boundary 
error,  regularization  and  subpixcl  displacement  error  estimation  in  the  design  and  imple¬ 
mentation  of  fast  and  robust  algorithms.  Interestingly,  the  multisensor  array  technology  was 
designed  by  NASA  to  help  reduce  the  costs  and  time  associated  with  removing,  calibrating, 
and  reinstalling  the  many  sensors  on  the  Space  Shuttle  launch  pads.  It  is  now  available  for 
licensing,  (see  http://www.edi.gatech.edu/nasa/Multisensor 

3.1  The  importance  of  phase  in  signals  and 
neural  network  implementation 

In  general,  with  reconstruction  from  magnitude-only  holograms,  the  reconstructed  object  is 
not  of  much  value  in  representing  the  original  object  whereas  reconstructions  from  phase- 
only  holograms  have  many  important  features  in  common  with  the  original  objects.  Many  of 
features  of  the  original  image  are  clearly  identifiable  in  the  phase-only  image  but  not  in  the 
magnitude-only  image.  Several  results  for  justifying  the  importance  of  phase  will  be  shown 
with  1-D  signal  and  multi-dimensional  signal  in  this  section. 

3.2  Wavelet  Superresolution 

Superresolution  produces  high-resolution  (HR)  image  from  a  set  of  low-resolution  (LR) 
frames.  The  relative  motions  in  successive  frames  are  estimated  and  used  for  aligning  the 
sample  points  in  each  frame  into  a  HR  grid.  There  are  various  types  of  models  [8]  used 
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to  represent  camera  motion,  namely,  translation,  rigid,  affine,  bilinear,  and  projective.  The 
most  general  model  is  the  projective  model  which  has  eight  motion  parameters.  After  reg¬ 
istering  all  LR  frames  into  a  HR  grid,  the  available  samples  distribute  nonuniformly.  Then 
the  wavelet  superresolution  algorithm  is  applied  in  order  to  get  the  HR  image. 

During  the  past  decade,  the  interest  in  seeking  enhancement  of  spatial  resolution 
leading  to  a  high-resolution  (HR)  image  from  a  sequence  of  degraded  (undersampled,  blurred, 
and  noisy)  low-resolution  (LR)  images  has  been  notable.  HR  images  are  required  in  many 
areas  including  remote  sensing,  military  surveillance  and  medical  diagnosis.  However,  it  is 
usually  not  possible  at  the  outset  to  achieve  the  desired  resolution  because  of  technology 
and  cost  constraints.  For  example,  the  technology  of  a  charge-coupled  device  (CCD)  is 
limited  by  factors  like  physical  dimension,  shot  noise  and  parasitic  effects.  In  applications 
like  astronomical  imaging,  the  reduced  size  and  weight  of  a  camera  in  a  spacecraft  or  satellite 
affect  its  quality.  The  need  to  perform  a  trade-off  between  size,  weight,  and  quality  of  the 
CCD  array  necessitates  the  design  of  superresolution  algorithms  to  obtain  the  desired  HR 
image. 


3.3  Multisensor  Array  Based  Superresolution 

An  image  acquisition  system  composed  of  an  array  of  sensors,  where  each  sensor  has  a  subar¬ 
ray  of  sensing  elements  of  suitable  size,  has  recently  been  popular  for  increasing  the  spatial 
resolution  with  high  signal-to-noise  ratio  beyond  the  performance  bound  of  technologies 
that  constrain  the  manufacture  of  imaging  devices.  Small  perturbations  around  the  ideal 
subpixel  locations  of  the  sensing  elements  (responsible  for  capturing  the  sequence  of  under- 
sampled  degraded  frames),  because  of  imperfections  in  fabrication,  limit  the  performance  of 
the  signal-processing  algorithms  for  processing  and  integrating  the  acquired  images  for  the 
desired  enhanced  resolution  and  quality.  With  the  objective  of  improving  the  performance 
of  the  signal-processing  algorithms  in  the  presence  of  the  ubiquitous  perturbation  errors  of 
displacements  around  the  ideal  subpixel  locations  (because  of  imperfections  in  fabrication)  in 
addition  to  noisy  observation,  the  errors-in-variables  or  the  total  least  squares  method  is  de- 
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ployed  here.  A  regularized  constrained  total  least  squares  (RCTLS)  solution  to  the  problem 
is  given  that  requires  the  minimization  of  a  nonconvex  and  nonlinear  cost  functional.  Sim¬ 
ulations  indicate  that  the  choice  of  the  regularization  parameter  influences  significantly  the 
quality  of  the  solution.  The  L-cnrve  method  is  deployed  to  select  the  theoretically  optimum 
value  of  the  regularization  parameter  instead  of  the  unsound  but  expedient  trial-and-error 
approach.  The  expected  superiority  of  this  RCTLS  approach  over  the  conventional  least 
squares  theory  based  algorithm  is  substantiated  by  example. 

4  Summary  of  the  Most  Important  Results 

This  research  focusses  on  two  superresolution  algorithms:  one  is  wavelet-based  and  the  other 
is  Delaunay  triangulation  based.  Preprocessing  is  performed  prior  to  the  implementation  of 
the  superresolution  algorithm  in  order  to  prevent  the  singularity  problem.  Attention  is  di¬ 
rected  to  the  selection  of  the  mother  wavelet  and  the  mother  scaling  function.  Then  the 
postprocessing  phase  incorporating  multiframe  noise  filtering  and  blur  removal  following  the 
implementation  of  the  superresolution  algorithm  is  developed.  The  overall  blind  superresolu¬ 
tion  procedure  is  illustrated  on  synthetic  sequences  and  also  on  video  (degraded  by  unknown 
blur  and  noise)  sequences  supplied  by  the  Air  Force  Research  Laboratory.  The  connection 
between  wavelet  superresolution  and  DTHR  algorithm  is  analyzed  and  a  promising  direction 
of  future  research  on  blind  robust  superresolution  by  using  the  second  generation  wavelet  is 
identified. 

The  contributions  of  this  research  also  includes  an  analysis  of  the  displacement  errors 
on  the  convergence  rate  of  the  iterative  approach  for  solving  the  transform  based  precon¬ 
ditioned  system  of  equations.  Subsequently,  it  is  established  that  the  use  of  the  MAP,  L2 
norm  or  H{  norm  regularization  functional  leads  to  a  proof  of  linear  convergence  of  the 
conjugate  gradient  method  in  terms  of  the  displacement  errors  caused  by  the  imperfect  sub- 
pixel  locations.  Results  of  simulation  support  the  analytical  results.  We  remark  that  when 
the  L2  norm  or  Hi  norm  regularization  functional  is  used,  the  corresponding  regularization 


matrices  under  the  Neumann  boundary  condition  can  be  diagonalized  by  the  discrete  cosine 
transform  matrix.  Thus  if  we  use  the  Neumann  boundary  condition  for  both  the  blurring 
matrix  Hl  and  the  regularization  matrix  then  the  coefficient  matrix  HJ  Hl  +  aPtP  can  be 
diagonalized  by  the  discrete  cosine  transform  matrix  and  hence  its  inversion  can  be  done 
with  three  2-dimensional  fast  cosine  transforms  (one  for  hireling  the  eigenvalues  of  the  coef¬ 
ficient  matrix,  two  for  transforming  the  right  hand  side  and  the  solution  vector;  see  [14]  for 
instance).  Thus  the  total  cost  of  solving  the  system  is  of  0(M\M2  log  Mi  M2)  operations. 
Here  g  is  an  Mi  x  M2  image  and  is  called  the  observed  high-resolution  image.  .  We  interlace 
the  low  resolution  images  to  form  an  M ]  x  M2  image  by  assigning 

g[L{n  1  -  1)  +  h,L(n2  -  1)  +  h\  =  ghi2[nl,n2\.  (1) 

We  have  showed  that  the  spectra  of  the  preconditioned  matrices  are  clustered  around  1  for 
sufficiently  small  e  [10]. 

Another  important  contribution  of  this  research  is  the  regularized  constrained  total 
least  squares  formulation  and  solution  of  the  high  resolution  image  reconstruction  problem 
with  multisensors  [11].  The  regularization  parameter  used  is  obtained  by  the  L-curve  method 
[15].  The  numerical  algorithm  developed  is  iterative  and  involves  a  two-step  minimization 
strategy  at  each  iteration.  Each  step,  in  turn,  involves  the  least-squares  solution  of  a  convex 
optimization  problem,  though  the  overall  problem  is  nonconvex.  A  characteristic  of  the 
numerical  strategy  developed  is  the  decrease  of  the  cost  functional  with  increase  in  the 
number  of  iterations. 
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7  Appendix  of  Figures 

Figure  1:  To  evaluate  the  importance  of  phase  information  in  the  presence  of  noise  in 
image  reconstruction.  Reconstructed  image  from  phase  only  information  has  many  important 
features  in  common  with  the  original  image.  Also,  reconstruction  from  noisy  phase  (and 
original  magnitude)  is  more  robust  to  error  than  reconstruction  from  noisy  magnitude  (and 
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original  phase).  Therefore,  phase  only  information  can  be  used  for  various  applications  like 
signal  reconstruction,  landmine  classifier,  and  so  on. 

Figure  2:  To  assess  the  importance  of  noise-corrupted,  degraded  and  partial  phase 
information  in  object  classification  by  artificial  neural  network  (ANN)  in  applications  like 
landmine  identification  and  classification.  Phase  information  is  more  error-tolerant  than 
magnitude  information  in  terms  of  signal  reconstruction.  The  number  of  input  neurons  can 
be  reduced  to  adequate  size  to  simplify  ANN  and  decrease  ANN  training  time.  Therefore, 
partial  phase  only  information  can  be  used  as  input  data  in  object  classification  by  ANN 

Figures  3:  Shows  the  overall  strategy  for  blind  robust  superresolution  using  either 
first  generation  wavelets  or  spatial  tesselation. 

Figures  4-9:  These  figures  show  selected  frames  from  a  real  video  sequence,  panoramic 
videomosaic  costructed  after  video  camera  motion  parameter  estimation,  and  the  results  fol¬ 
lowing  implementation  of  superresolution  algorithm  (Delaunay  triangulation  based)  on  two 
selected  ROIs  in  the  mosaic. 

Figures  10-12:  Multiple  images  mean  image  sequences  of  monochrome  or  multi- 
spectral  images,  here.  Image  sequences  are  produced  from  snapshots  of  an  object  or  scene, 
or  from  the  multisensor  array  system  while  multispectral  images  are  acquired  by  multiple 
sensors  with  wavelength  optical  filters.  For  background  and  notation,  see  the  original  paper 
by  Bose  and  Boo  [16]. 
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Original  image 
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from  magnitude 
with  zero  phase  by 
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Figure  1:  Original  image  and  reconstructed  images 
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Figure  2:  Overall  neural  network  for  object  classification 
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Box  A 


Figure  3:  The  HR  image  reconstruction  can  be  divided  into  two  parts:  interpolation  and 
restoration.  The  image  registration  is  the  inverse  geometric  transformation.  It  transforms 
LR  samples  into  a  HR  grid.  The  multiframe  blur  identification  part  estimates  the  point 
spread  function  (PSF)  of  blur  from  the  set  of  LR  images,  either  directly  or  indirectly,  and 
then  used  in  the  restoration  part. 


Figure  4:  Some  LR  frames  in  the  real  video  sequence 
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Figure  5:  Video  mosaic  with  the  boundary  of  ROIs 
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Figure  6:  Sample  of  LR  frames  in  ROI-1 
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Figure  7:  HR  image  of  ROI-1 
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Figure  8:  Sample  of  LR  frames  in  ROI-2 
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Figure  9:  HR  image  of  ROI-2 
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Figure  10:  Image  formation 


by  using  multiple  CCD  sensor  arrays 


Figure  11:  The  relationship  between  high  and  low  resolution  image  sensors 


22 


Figure  12:  Example  of  HR  image 
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ction  from  a  set  of  degraded  LR  images  when 
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